Method and apparatus of video coding with affine motion compensation

ABSTRACT

A method and apparatus for video encoding and decoding with affine motion compensation are disclosed. An embodiment of the method receives input data associated with a current block coded or to be coded in an affine mode. Two or more motion vectors of a neighboring block are retrieved from a buffer, where at least one of the retrieved motion vectors is not corresponding to a sub-block in a first neighboring block row or a first neighboring block column. The method derives an affine candidate including affine motion vectors using the retrieved two or more motion vectors, and encodes or decodes the current block by predicting the current block using one or more motion compensated blocks by the derived affine candidate. The buffer for affine candidate derivation stores selective motion vectors less than all motion vectors of coded blocks in the current picture.

CROSS REFERENCE TO RELATED APPLICATIONS

The present invention claims priority to PCT Patent Application, SerialNo. PCT/CN2016/076360, filed on Mar. 15, 2016, entitled “AffinePrediction for Video Coding” and U.S. Provisional Patent Application,Ser. No. 62/437,757, filed on Dec. 22, 2016, entitled “Affine PredictionBuffer Management for Video Coding”. The PCT Patent Application and theU.S. Provisional Patent Application are hereby incorporated by referencein their entireties.

FIELD OF THE INVENTION

The present invention relates to video coding with affine motioncompensation. In particular, the present invention relates to buffermanagement for a video coding system implementing affine motioncompensation and techniques to enable adaptive motion vector resolutionfor affine motion compensation.

BACKGROUND AND RELATED ART

Video data requires a large storage space to buffer intermediate dataduring encoding and decoding. Along with the growing high resolution andhigher frame rates, as well as more powerful coding techniques aredeveloped to achieve better coding performance, the storage requirementfor video encoder and decoder increase significantly. One of the newlydeveloped coding techniques is affine motion prediction andcompensation, which effectively tracks more complicated motion such asrotation, zooming, and the deformation of moving objects. Inter framemotion prediction methods applied in the recently developed codingstandards such as High Efficiency Video Coding (HEVC) only considertwo-dimensional (2D) translational motion, where all the pixels in anarea of interest follow the same motion direction and magnitude. Affinemotion prediction is capable of describing 2D block rotations andscaling according to a four-parameter affine model. Affine motionprediction is also capable of capturing 2D deformations according to asix-parameter affine model which transforms a square or a rectangle intoa parallelogram. There are two major modes for affine motion predictionproposed in the literatures, including affine Merge mode and affineInter mode. Affine Merge mode allows the inheritance of affine motioninformation from a spatially neighboring block whereas affine Inter modeconstructs several most probable candidates by combining motioninformation of spatially neighboring blocks. Affine Inter mode is alsoreferred to affine advance motion vector prediction (AMVP) mode.

Motion occurs across pictures along temporal axis can be described by afour-parameter affine motion model as shown in Equation (1). AssumingA(x, y) is an original pixel at location (x, y) under consideration, andA′(x′, y′) is a corresponding reference pixel at location (x′, y′) in areference picture for the original pixel A(x, y).

x′=a0*x+a1*y+a2, and

y′=−a1*x+a0*y+a3.   (1)

where a0, a1, a2 and a3 are the four parameters in the four-parameteraffine motion model.

The motion vector (v_(x), v_(y)) between this original pixel A(x,y) andits corresponding reference pixel A′(x′,y′) in a block coded in anaffine mode is described as:

v _(x)=(1−a0)*x−a1*y−a2, and

v _(y)=(1−a0)*y+a1*x−a3.   (2)

An exemplary four-parameter affine motion model is shown in FIG. 1A. Twocorner pixels 110 and 112 are located at upper-left and upper-rightcorners of a current block 102, and these two corner pixels are alsocalled control points for the current block 102 in the four-parameteraffine motion model. Motion vectors Mv0 and Mv1 of the two controlpoints 110 and 112 map the current block 102 to a reference block 104 ina reference picture. The motion vector field of each pixel A(x,y) in thecurrent block 102 may be derived based on the motion vectors Mv0 and Mv1of the control points 110 and 112 according to Equation (3).

$\begin{matrix}\left\{ \begin{matrix}{v_{x} = {{\frac{\left( {v_{1\; x} - v_{0\; x}} \right)}{w}x} - {\frac{\left( {v_{1\; y} - v_{0\; y}} \right)}{w}y} + v_{0\; x}}} \\{v_{y} = {{\frac{\left( {v_{1\; y} - v_{0\; y}} \right)}{w}x} + {\frac{\left( {v_{1\; x} - v_{0\; x}} \right)}{w}y} + v_{0\; y}}}\end{matrix} \right. & (3)\end{matrix}$

where (v_(0x), v_(0y)) represents the motion vector Mv0 at theupper-left corner 110, (v_(1x), v_(1y)) represents the motion vector Mv1at the upper-right corner 112, and w represents a width of the currentblock. For block-based affine motion compensation, when the motionvectors Mv0 and Mv1 of the two control points are decoded, the motionvector of each 4×4 block of the current block 102 can be determinedaccording to Equation (3). In other words, the four-parameter affinemotion model for the current block 102 can be specified by the twomotion vectors Mv0 and Mv1 at the two control points. Furthermore, whilethe upper-left corner and the upper-right corner of the block are usedas the two control points, other two control points may also be used.

A six-parameter affine motion model can be described by Equation (4). Inthis model, a total of six parameters a0, a1, a2, b0, b1 and b2 andthree control points are used. For each pixel A(x, y), the motion vector(v_(x), v_(y)) between this pixel A(x, y) and its correspondingreference pixel A′(x′, y′) is shown in Equation (5).

x′=a0+a1*x+a2*y, and

y′=b0+b1*x+b2*y.   (4)

v _(x)=(a1−1)*x+a2*y+a0, and

v _(y)=(b2−1)*y+b1*x+b0.   (5)

The motion vector for each pixel predicted by the six-parameter affinemotion model is also location dependent. FIG. 1B illustrates an exampleof affine motion compensation according to the six-parameter affinemotion model, where a current block 122 is mapped to a reference block124 in a reference picture. The correspondences between three cornerpixels 130, 132, and 134 of the current block 122 and three cornerpixels of the reference block 124 can be determined by the three arrowsas shown in FIG. 1B. The six parameters for the affine motion model canbe derived based on three known motion vectors Mv0, Mv1, Mv2 of theupper-left, upper-right, and lower-left control points of the currentblock 122. Parameter derivation for the affine motion model is known inthe field and the details are omitted here.

Various implementations of affine Inter mode and affine Merge mode havebeen discussed, for example, an affine flag is used to indicate whetherthe affine Inter mode is applied, and this affine flag is signaled foreach Inter-coded coding unit (CU) when the CU is equal to or larger than16×16. A candidate motion vector predictor (MVP) pair list isconstructed for a current CU using valid neighboring coded blocks if thecurrent CU is coded or to be coded in affine Inter mode. FIG. 2illustrates an example of the candidate MVP pair derivation for acurrent block 20 coded in affine Inter mode or affine Merge mode. Asshown in FIG. 2, a motion vector predictor (MVP) for the motion vectorMv0 at an upper-left control point of the current block 20 is selectedfrom motion vectors of upper-left neighboring coded blocks A0, A1, orA2; and a MVP for Mv1 at an upper-right control point of the currentblock 20 is selected from motion vectors of upper-right neighboringcoded blocks B0 and B1. An MVP index for the candidate MVP pair list issignaled in a video bitstream and motion vector differences (MVDs) ofthe two control points are coded in the video bitstream.

For a current block 20 coded in Merge mode, five neighboring codedsub-blocks C0 (referred as the left-bottom block), B0 (referred as thetop-right block), B (referred as the upper-right corner block), C1(referred as the lower-left corner block), and A0 (referred as theupper-left corner block) in FIG. 2 are sequentially checked todetermines whether any of the neighboring coded sub-blocks is coded inaffine Inter mode or affine Merge mode. The current block 20 in thisexample is a prediction unit (PU). An affine flag is signaled toindicate whether the current block 20 is coded in affine Merge mode onlyif any of the neighboring coded sub-blocks is coded in affine Inter modeor affine Merge mode. When encoding or decoding the current block 20according to affine Merge mode, a first available affine-codedneighboring block is determined by selecting from the five neighboringcoded sub-blocks. The first available affine-coded neighboring blockincluding the selected neighboring coded sub-block is used to derive anaffine Merge candidate. The affine Merge candidate is used for derivinga predictor in a reference picture for the current block. The selectionorder for selecting one of the neighboring coded sub-blocks is fromleft-bottom block, top-right block, upper-right corner block, lower-leftcorner block to upper-left corner block (C0→B→B1→C1→A0) as shown in FIG.2. The affine Merge candidate for the current block 20 is derived fromMVs at the control points of the first available affine-codedneighboring block, for example, the MVs of a top-left N×N sub-block anda top-right N×N sub-block of the first available affine-codedneighboring block are used to derive the affine Merge candidate if afour-parameter affine motion model is applied. The MV of a bottom-leftN×N sub-block of the first available affine-coded neighboring block isalso used to derive the affine Merge candidate when a third controlpoint is included for a six-parameter affine motion model.

BRIEF SUMMARY OF THE INVENTION

A method and apparatus for video encoding and decoding with affinemotion compensation in a video coding system are disclosed. Embodimentsof a video encoder according to the present invention receive input dataassociated with a current block in a current picture and embodiments ofa video decoder according to the present invention receive a videobitstream corresponding to compressed data including the current blockin the current picture. The current block is coded or to be coded in anaffine mode according to an affine motion model. Various embodiments ofthe present invention reduce the buffer requirement of a temporal bufferfor affine candidate derivation. The temporal buffer stores selectivemotion vectors less than all motion vectors of previous coded blocks inthe current picture. Embodiments of the method retrieve two or moremotion vectors (MVs) of a neighboring block from the temporal buffer,where at least one of the retrieved motion vectors is not correspondingto a sub-block in a first neighboring N×N block row closest to a topboundary of the current block or a first neighboring N×N block columnclosest to a left boundary of the current block, and N×N is a block sizefor storing one motion vector in the temporal buffer. The method furtherderives an affine candidate including affine motion vectors using theretrieved two or more motion vectors of the neighboring block, andencodes or decodes the current block by predicting the current blockusing one or more motion compensated blocks derived by the derivedaffine candidate. Each of the affine motion vectors predicts a movementbetween a point of the current block and a corresponding point of one ormore motion compensated blocks.

In some embodiments, the temporal buffer stores MVs of the firstneighboring N×N block row and a second neighboring N×N block row thatare closest to the top boundary of the current block, and the temporalbuffer stores MVs of the first neighboring N×N block column and a secondneighboring N×N block column that are closest to the left boundary ofthe current block.

In one embodiment, the retrieved MVs includes first and second MVs for afour-parameter affine motion model, for example, the first and secondMVs are replacing MVs replacing original MVs at an upper-left corner andan upper-right corner of the neighboring block respectively if theneighboring block is above or adjacent to an upper-left corner of thecurrent block. The first MV is a replacing MV replacing an original MVat an upper-left corner of the neighboring block and the second MV is anoriginal MV at an upper-right corner of the neighboring block if theneighboring block is on the left side of the current block. In anembodiment, the first and second MVs correspond to first and secondsub-blocks in the neighboring block, and a first affine MV of the affineMVs is derived using the first and second MVs, a pixel position of thecurrent block relative to the current picture, a pixel position of thefirst sub-block relative to the current picture, and a width of theneighboring block. A second affine MV of the affine MVs is derived usingthe first and second MVs, the width of the neighboring block, and awidth of the current block.

In another embodiment, the retrieved MVs includes first, second andthird MVs for a six-parameter affine motion model, for example, thethree MVs are for first, second and third sub-blocks in the neighboringblock, and a first affine MV is derived using at least two of the threeMVs, a pixel position of the current block relative to the currentpicture, a pixel position of the first sub-block relative to the currentpicture, and a width of the neighboring block. A second affine MV isderived using the first and second motion vectors, the width of theneighboring block, and a width of the current block, and a third affineMV is derived using the first and third motion vectors and at least oneof a height of the neighboring block and a height of the current block.In this embodiment, the first, second MVs are replacing MVs replacingoriginal MVs at an upper-left corner and an upper-right corner of theneighboring block, and the third MV is an original MV at a lower-leftcorner of the neighboring block if the neighboring block is above thecurrent block. The first, second, and third MVs are replacing MVsreplacing original MVs at an upper-left corner, upper-right corner, andlower left corner of the neighboring block if the neighboring block isadjacent to an upper-left corner of the current block. The first andthird MVs are replacing MVs replacing original MVs at an upper-leftcorner and lower-left corner of the neighboring block, and the second MVis an original MV at an upper-right corner of the neighboring block ifthe neighboring block is on the left side of the current block.

The current block is predicted by the motion compensated blocks by theaffine motion vectors in the affine candidate according to afour-parameter affine motion model with two control points or accordingto a six-parameter affine motion model with three control points. Thetemporal buffer stores the selective motion vectors, for example, thetemporal buffer stores MVs of M N×N block rows above the current block,and M is less than a largest coding unit height divided by N(CTU_height/N). In another embodiment, the temporal buffer stores MVs ofK N×N block columns on a left side of the current block, and K is lessthan a largest coding unit width divided by N (CTU_width/N).

In some embodiments of the method, the retrieved MVs are original MVs attwo or more control points of the neighboring block. The control pointsinclude at least two of an upper-left corner, an upper-right corner, abottom-right corner, and a bottom-left corner of the neighboring block.In an embodiment, the temporal buffer stores MVs of 2 N×N block rows and2 N×N block columns, including original MVs of a first neighboring N×Nblock row, a first neighboring N×N block column, a top N×N block row setand a left-most N×N block column set. The first neighboring N×N blockrow is a last row in upper neighboring blocks that is closest to the topboundary of the current block, the first neighboring N×N block column isa last column in left neighboring blocks that is closest to the leftboundary of the current block, the top N×N block row set is the firstrows in the upper neighboring blocks, and the left-most N×N block columnset is the first columns in the left neighboring blocks.

In yet another embodiment, the method comprises receiving input dataassociated with a current block coded or to be coded in an affine mode,calculating and storing affine parameters for a plurality of codedblocks in the current picture, retrieving the affine parameter of one ormore coded block from the temporal buffer corresponding to a neighboringblock of the current block to derive an affine candidate includingaffine MVs. The current block is encoded or decoded by predicting thecurrent block using one or more motion compensated blocks by the derivedaffine candidate. Each of the affine motion vectors predicts a movementbetween a point of the current block and a corresponding point of theone or more motion compensated blocks.

The affine parameters in an embodiment include a horizontal directionmotion vector (MV) offset and one motion vector in the neighboring blockwhen the affine motion model is a four-parameter affine motion modelusing two of an upper-left corner, an upper-right corner, an lower-leftcorner, and a lower-right corner as the control points, or the affineparameters include a vertical direction motion vector (MV) offset andone motion vector in the neighboring block when the affine motion modelis a four-parameter affine motion model using two of an upper-leftcorner, an upper-right corner, a lower-left corner, and a lower-rightcorner as control points. The affine parameters in another embodimentinclude a horizontal direction MV offset, a vertical direction MVoffset, and a motion vector in the neighboring block when the affinemotion model is a six-parameter affine motion model. An example of themotion vector in the neighboring block is a motion vector at anupper-left corner, upper-right corner, lower-right corner, or lower-leftcorner of the neighboring block. The affine parameters in anotherembodiment include a scaled MV offset for the coded block. The affineparameters in yet another embodiment include two or three affine motionvectors representing motion vectors at two or three control points, andthe temporal buffer stores the two or three affine motion vectors forthe coded block.

Another embodiment of the method comprises receive input data associatedwith a current block coded or to be coded in an affine mode, retrievingtwo or more MVs of a valid neighboring block from a temporal buffer forthe current block, deriving an affine candidate including affine MVsusing the retrieved two or more MVs of the valid neighboring block, andencoding or decoding the current block by predicting the current blockusing one or more compensated blocks by the derived affine candidate.The valid neighboring block does not include a neighboring blockadjacent to an upper-left corner of the current block. The temporalbuffer stores MVs of one neighboring N×N block row and one neighboringN×N block column of the current block, where N×N is a block size forstoring one MV in the temporal buffer. The affine MVs predict a movementbetween a point of the current block and a corresponding point of theone or more motion compensated blocks.

Aspects of the disclosure further provide an apparatus comprising one ormore electronic circuits configured for performing the video codingmethods with affine motion compensation. Other aspects and features ofthe invention will become apparent to those with ordinary skill in theart upon review of the following descriptions of specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates four-parameter affine prediction mapping a currentblock to one or more compensated blocks according to two control points.

FIG. 1B illustrates six-parameter affine prediction mapping a currentblock to one or more compensated blocks according to three controlpoints.

FIG. 2 illustrates an example of affine candidate derivation based onneighboring coded blocks for affine Inter mode or affine Merge mode.

FIG. 3 illustrates an example of encoding or decoding a current blockusing information from a neighboring block according to affine Mergemode.

FIGS. 4A and 4 B illustrate two examples of storing MVs of twoneighboring block rows and two neighboring block columns for affinecandidate derivation according to embodiments of the present invention.

FIG. 5 illustrates an exemplary flowchart for a video coding system withaffine motion compensation incorporating an embodiment of the presentinvention.

FIG. 6 illustrates an example of storing original MVs at control pointsof neighboring blocks for affine candidate derivation according to anembodiment of the present invention.

FIG. 7 illustrates an exemplary flowchart for a video coding system withaffine motion compensation incorporating embodiments of the presentinvention.

FIG. 8 illustrates an example of storing MVs of a closest neighboringblock row and closest neighboring block column for affine candidatederivation according to an embodiment of the present invention.

FIG. 9 illustrates an exemplary system block diagram for a video encoderimplementing an affine motion prediction and compensation according tovarious embodiments of the present invention.

FIG. 10 illustrates an exemplary system block diagram for a videodecoder implementing affine motion compensation according to variousembodiments of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

It will be readily understood that the components of the presentinvention, as generally described and illustrated in the figures herein,may be arranged and designed in a wide variety of differentconfigurations. Thus, the following more detailed description of theembodiments of the systems and methods of the present invention, asrepresented in the figures, is not intended to limit the scope of theinvention, as claimed, but is merely representative of selectedembodiments of the invention.

Reference throughout this specification to “an embodiment”, “someembodiments”, or similar language means that a particular feature,structure, or characteristic described in connection with theembodiments may be included in at least one embodiment of the presentinvention. Thus, appearances of the phrases “in an embodiment” or “insome embodiments” in various places throughout this specification arenot necessarily all referring to the same embodiment, these embodimentscan be implemented individually or in conjunction with one or more otherembodiments. Furthermore, the described features, structures, orcharacteristics may be combined in any suitable manner in one or moreembodiments. One skilled in the relevant art will recognize, however,that the invention can be practiced without one or more of the specificdetails, or with other methods, components, etc. In other instances,well-known structures, or operations are not shown or described indetail to avoid obscuring aspects of the invention.

A decoder complied with the HEVC standard down-samples the decodedmotion vectors of each PU coded by inter prediction including AMVP modeand Merge mode with a 16:1 ratio, and the decoder stores thedown-sampled motion vectors in a buffer for MVP derivation for followingblocks in a current picture and following pictures. A motion vector ofthe top-left 4×4 block in every 16×16 block is stored in the buffer andthe stored motion vector represents the motion vector of the entire16×16 block. The motion vector precision may be 1/64 pixel precision anda predictor is generated by applying a high accuracy Discrete CosineTransform Interpolation Filter (DCTIF). The high accuracy motion vectorfield is then clipped to 1/8 pixel precision before storing to thebuffer.

In order to process a block coded in affine Merge mode, a firstavailable affine-coded neighboring block is determined according to theselection order of neighboring coded sub-blocks A, B, C, D and E shownin FIG. 3. FIG. 3 illustrates an example of encoding or decoding acurrent block 30 using information of a neighboring block according toaffine Merge mode. In this example, two neighboring coded sub-blocks Band E of the current block 30 are coded in an affine mode. A neighboringblock 32 including the neighboring coded sub-block B may be eitheraffine Inter coded block or affine Merge coded block, similarly, aneighboring block 34 including the neighboring coded sub-block E may beeither affine Inter coded block or affine Merge coded block. The firstavailable affine-coded neighboring block in this example is theneighboring block 32. The affine Merge candidate for predicting thecurrent block 30 is derived from the motion vector V_(B0) of a top-leftN×N block 322, the motion vector V_(B1) of a top-right N×N block 324 ofthe first available affine-coded neighboring block 32 if two controlpoints are used in the affine mode. The affine Merge candidate isfurther derived from the motion vector V_(B2) of a bottom-left N×N block326 of the first available affine-coded neighboring block 32 if threecontrol points are used in the affine mode. N×N is the smallest blocksize for storing an MV in the temporal MV buffer, for example, N equalsto 4. In HEVC systems, only the motion vectors of the neighboring 4×4block row and the neighboring 4×4 block column of the current codingunit (CU) or coding tree unit (CTU) and the motion vectors of thecurrent CTU are stored. The CTU is also a term to define a largestcoding unit (LCU) allowed in the video coding system. All other motionvectors are either discarded or down-sampled in ratio of 16:1 and storedin the buffer. A video coding system with affine motion compensationrequires additional MV buffers to store motion vectors of neighboringcoded blocks for affine candidate derivation.

Embodiments of buffer management for a video coding system are describedin the following which implements affine motion compensation withreduced buffer requirements.

First Embodiment

In a first embodiment, a current block in a current picture is encodedor decoded by affine motion compensation according to an affinecandidate including affine motion vectors. Each of the affine motionvectors predicts a motion vector at a control point of the current blockas the affine motion vectors are used to predict a movement between apoint of the current block and a corresponding point of an affine motioncompensated block. A temporal MV buffer in the first embodiment storesmotion vectors of a single neighboring N×N block row and a singleneighboring N×N block column of the current block. N×N represents thesmallest block size to store a motion vector in the temporal MV buffer,for example N is 4. In this embodiment, the number of MVs stored in thetemporal MV buffer is the same as a conventional HEVC temporal MV bufferfor MVP derivation. When the derivation of the affine candidate for thecurrent block requires a motion vector at a control point of anaffine-coded neighboring block, and the control point is not in thecurrent CTU and not in the neighboring N×N block row or the neighboringN×N block column of the current block, a replacing motion vector isretrieved from a buffer to derived a corresponding affine motion vectorinstead of an original motion vector at the control point of theaffine-coded neighboring block. The replacing motion vector is adown-sampled motion vector stored in the buffer, and this replacingmotion vector represents the motion of the entire 16×16 block. Thereplacing motion vector may just be a motion vector used by a sub-blocknear the control point. For example, the replacing motion vector iscorresponding to a sub-block that is belong to the same 16×16 block asthe control point. Affine motion vectors in an affine candidate derivedaccording to the first embodiment are typically not precise motionvector predictors since the down-sampled motion vectors sometimes do notreflect the real motion at control points of the neighboring block.

Second Embodiment

A second embodiment stores more motion vectors in the temporal MV bufferby storing MVs of M neighboring N×N block rows and MVs of K neighboringN×N block columns. In this embodiment, M and K are integers greater thanor equal to 2, and N is an integer greater than 1. Instead of storingall motion vectors of the coded blocks in the current picture in thetemporal MV buffer, selective motion vectors less than all motionvectors of the coded blocks in the current picture are stored. Theselective motion vectors are MVs of M block rows and K block columns,where the number of block rows M should be less than the height of alargest coding tree unit divided by N (1<M<CTU_height/N). The N×N blockrefers to the smallest block size for storing an MV in the temporal MVbuffer. In an example of the second embodiment, both M and K are 2 asshown in FIG. 4A and FIG. 4B. The temporal buffer in this example storesmotion vectors of first and second neighboring N×N block rows closest toa top boundary of the current block and the temporal buffer also storesmotion vectors of first and second neighboring N×N block columns closestto a left boundary of the current block. FIG. 4A and FIG. 4B illustratetwo examples of affine candidate derivation for a current block coded inaffine mode using two neighboring block rows and two neighboring blockcolumns. If motion vectors at control points of a neighboring block arenot stored in the temporal MV buffer as the corresponding sub-blocks arenot in the two neighboring block rows and not in the two neighboringblock columns, some replacing motion vectors are retrieved to deriveaffine motion vectors in the affine candidate.

In the example shown in FIG. 4A, neighboring coded blocks 41, 42 and 43of a current block 40 are all coded in affine mode. Replacing motionvectors V_(B0′) and V_(B1′) of sub-blocks 423 and 425 in a secondneighboring block row above the current block 40 are used to deriveaffine motion vectors in the affine candidate instead of original motionvectors V_(B0) and V_(B1) of sub-blocks 422 and 424 in a first row ofthe neighboring coded block 42. Replacing motion vectors V_(E0′),V_(E1′) and V_(E2′) of sub-blocks 433, 435, 437 in either the secondneighboring block row above the current block 40 or a second neighboringblock column on the left of the current block 40 are used to deriveaffine motion vectors instead of original motion vectors V_(E0), V_(E1)and V_(E2) of sub-blocks 432, 434 and 436. Replacing motion vectorsV_(A0′) and V_(A2′) of sub-blocks 413 and 415 in the second neighboringblock column on the left of the current block 40 are used to derive theaffine motion vectors instead of original motion vectors V_(A0) andV_(A2) of sub-block 412 and 416 in a first column of the neighboringcoded block 41. Original motion vectors V_(B2) and V_(A1) of sub-blocks426 and 414 are used to derive the affine motion vectors for the currentblock 40 because these two sub-blocks 426 and 414 are within the firstand second neighboring block rows or the first and second neighboringblock columns of the current block 40. The derived affine motion vectorsin the affine candidate are used to predict a movement between points ofthe current block 40 and corresponding points of one or more compensatedblocks using an affine motion model.

FIG. 4B is an alternative solution to the affine candidate derivationmethod illustrated in FIG. 4A. In FIG. 4B, when deriving an affinecandidate for encoding or decoding a current block 44, all the originalmotion vectors at control points of a neighboring affine-coded block arereplaced by motion vectors of other sub-blocks in first and secondneighboring block rows and first and second neighboring block columns ofthe current block 44. The control points for an affine-coded blockpredicted by a six-parameter affine motion model include upper-leftcorner, upper-right corner, and lower-left corner of the affine-codedblock. Replacing motion vectors V_(B0′), V_(B1′) and V_(B2′) ofsub-blocks 463, 465 and 467 in a neighboring affine-coded block 46 areused to derive the affine motion vectors instead of using originalmotion vectors V_(B0), V_(B1) and V_(B2) of sub-blocks 462, 464 and 466.Replacing motion vectors V_(E0′), V_(E1′) and V_(E2′) of sub-blocks 473,475 and 477 in a neighboring affine-coded block 47 are used to derivethe affine motion vectors instead of original motion vectors V_(E0),V_(E1) and V_(E2) of sub-blocks 472, 474 and 476. Replacing motionvectors V_(A0′), V_(A1′) and V_(A2′) of sub-blocks 453, 455 and 457 in aneighboring affine-coded block 45 are used to derive the affine motionvectors instead of original motion vectors V_(A0), V_(A0) and V_(A2) ofsub-blocks 452, 454 and 456. In general, other positions in the Mneighboring block rows and the K block columns may be used for affinecandidate derivation in this embodiment.

Without loss of generality, only the affine candidate derivation methodillustrated in FIG. 4A is further described in the following. In a firstexample, an affine candidate including three affine motion vectors Mv0,Mv1 and Mv2 are derived for predicting the current block 40 using asix-parameter affine motion model. In the first example, the affinemotion vectors of the affine candidate are derived from the neighboringaffine-coded block 42. Affine motion vector Mv0=(V0_x, V0_y) for a firstcontrol point at an upper-left corner is derived by Equation (6).

$\begin{matrix}{{V_{0{\_ x}} = {V_{B\; 0^{\prime}{\_ x}} + {\left( {V_{B\; 2^{\prime}{\_ x}} - V_{B\; 0^{\prime}{\_ x}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 0^{\prime}{\_ Y}}} \right)/\left( {2*N} \right)}} + {\left( {V_{B\; 1^{\prime}{\_ x}} - V_{B\; 0^{\prime}{\_ x}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 0^{\prime}{\_ X}}} \right)/{RefPU\_ width}}}}},{{V_{0{\_ y}} = {V_{B\; 0^{\prime}{\_ y}} + {\left( {V_{B\; 2^{\prime}{\_ y}} - V_{B\; 0^{\prime}{\_ y}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 0^{\prime}{\_ Y}}} \right)/\left( {2*N} \right)}} + {\left( {V_{B\; 1^{\prime}{\_ y}} - V_{B\; 0^{\prime}{\_ y}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 0^{\prime}{\_ X}}} \right)/{RefPU\_ width}}}}};}} & (6)\end{matrix}$

where motion vectors V_(B0′)=(V_(B0′) _(_) _(x), V_(B0′) _(_) _(y)),V_(B1′)=(V_(B1′) _(_) _(x), V_(B1′) _(_) _(y)), and V_(B2)=(V_(B2) _(_)_(x), V_(B2) _(_) _(y)) are the three motion vectors retrieved from Mneighboring N×N block rows and K neighboring N×N block columns. It isalso to be understood that these motion vectors can be replaced bycorresponding motion vectors of any other selected sub-blocks in the Mneighboring N×N block rows and K neighboring N×N block columns of thecurrent block. A coordinate (posCurPU_X, posCurPU_Y) represents a pixelposition of an upper-left corner sample of the current block 40 relativeto an upper-left corner sample of the current picture. A coordinate(posB0′_X, posB0′_Y) represents a pixel position of an upper-left cornersample of the sub-block 422 relative to the upper-left corner sample ofthe current picture. RefPU_width represents the width of the neighboringblock 42. Affine motion vectors Mv1=(V₁ _(_) _(x), V₁ _(_) _(y)) andMv2=(V₂ _(_) _(x), V₂ _(_) _(y)) for second and third control points atupper-right and lower-left corner respectively are derived by Equation(7).

V ₁ _(_) _(x) =V ₀ _(_) _(x)+(V _(B1′) _(_) _(x) −V _(B0′) _(_)_(x))*PU_width/RefPU_width,

V ₁ _(_) _(y) =V ₀ _(_) _(y)+(V _(B1′) _(_) _(y) −V _(B0′) _(_)_(y))*PU_width/RefPU_width;

V ₂ _(_) _(x) =V ₀ _(_) _(x)+(V _(B2) _(_) _(x) −V _(B0′) _(_)_(x))*PU_height/(2*N),

V ₂ _(_) _(y) =V ₀ _(_) _(y)+(V _(B2) _(_) _(y) −V _(B0′) _(_)_(y))*PU_height/(2*N);   (7)

where PU_width and PU_height represent the width and height of thecurrent block 40.

In a second example, an affine candidate including two affine motionvectors Mv0 and Mv1 is derived for predicting the current block 40 usinga four-parameter affine motion model. The second example also uses themotion vectors from the neighboring affine-coded block 42 to derive theaffine candidate. Affine motion vectors Mv0=(V0_x, V0_y) and Mv1=(V1_x,V1_y) for first and second control points at upper-left and upper-rightcorners respectively are derived by Equation (8).

$\begin{matrix}{{{V_{0{\_ x}} = {V_{B\; 0^{\prime}{\_ x}} - {\left( {V_{B\; 1^{\prime}{\_ y}} - V_{B\; 0^{\prime}{\_ y}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 0^{\prime}{\_ Y}}} \right)/{RefPU\_ width}}} + {\left( {V_{B\; 1^{\prime}{\_ x}} - V_{B\; 0^{\prime}{\_ x}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 0^{\prime}{\_ X}}} \right)/{RefPU\_ width}}}}},{{V_{0{\_ y}} = {V_{B\; 0^{\prime}{\_ y}} + {\left( {V_{B\; 1^{\prime}{\_ x}} - V_{B\; 0^{\prime}{\_ x}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 0^{\prime}{\_ Y}}} \right)/{RefPU\_ width}}} + {\left( {V_{B\; 1^{\prime}{\_ y}} - V_{B\; 0^{\prime}{\_ y}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 0^{\prime}{\_ X}}} \right)/{RefPU\_ width}}}}};}}\mspace{20mu} {{V_{1{\_ x}} = {V_{0{\_ x}} + {\left( {V_{B\; 1^{\prime}{\_ x}} - V_{B\; 0^{\prime}{\_ x}}} \right)*{{PU\_ width}/{RefPU\_ width}}}}},\mspace{20mu} {V_{1{\_ y}} = {V_{0{\_ y}} + {\left( {V_{B\; 1^{\prime}{\_ y}} - V_{B\; 0^{\prime}{\_ y}}} \right)*{{PU\_ width}/{{RefPU\_ width}.}}}}}}} & (8)\end{matrix}$

Considering the line buffer of storing motion vectors of top CTUs ismuch larger than the column buffer of storing motion vectors of a leftCTU, in one example, there is no need to constrain the value of K inthis second embodiment, all motion vectors of the left CTU are stored bysetting K equals to the width of a largest coding unit divided by N(K=CTU_width/N).

Third Embodiment

In a third embodiment, affine parameters or control points for everyfixed size block or every CU are deliberately stored. In Equation (3),the motion vector Mv0=(V_(0x), V_(0y)) of a top-left N×N sub-block andthe motion vector Mv1=(V_(1x), V_(1y)) of a top-right N×N sub-block areused to derive motion vectors of all N×N sub-blocks in the currentblock. The current block is either a coding unit (CU) or a predictionunit (PU). The derived motion vectors can be represented by the motionvector Mv0 plus a position dependent MV offset. From Equation (3), forderiving a motion vector for an N×N sub-block at position (x,y), ahorizontal direction MV offset H_MV_offset and a vertical direction MVoffset V_MV_offset are shown in Equation (9).

H_MV_offset=(V _(1x) −V _(0x))*N/w, (V _(1y) −V _(0y))*N/w;

V_MV_offset=−(V _(1y) −V _(0y))*N/w, (V _(1x) −V _(0x))*N/w.   (9)

For a six-parameter affine motion model, the motion vector Mv0=(V_(0x),V_(0y)) of a top-left N×N sub-block, the motion vector Mv1=(V_(1x),V_(1y)) of a top-right N×N sub-block, and the motion vector Mv2=(V_(2x),V_(2y)) of a bottom-left N×N sub-block are used to derive motion vectorsof all N×N sub-blocks in the current block. The motion vector field ofeach pixel A(x,y) in the current block may be derived based on the threemotion vectors Mv0, Mv1 and Mv2 according to Equation (10).

$\begin{matrix}\left\{ \begin{matrix}{v_{x} = {{\frac{\left( {v_{1\; x} - v_{0\; x}} \right)}{w}x} - {\frac{\left( {v_{2\; x} - v_{0\; x}} \right)}{h}y} + v_{0\; x}}} \\{v_{y} = {{\frac{\left( {v_{1\; y} - v_{0\; y}} \right)}{w}x} + {\frac{\left( {v_{2\; y} - v_{0\; y}} \right)}{h}y} + v_{0\; y}}}\end{matrix} \right. & (10)\end{matrix}$

Similarly, for deriving a motion vector (V_(x), V_(y)) of an N×Nsub-block at position (x,y) according to the six-parameter affine motionmodel, a horizontal direction MV offset H_MV_offset and a verticaldirection MV offset H_MV_offset are shown in Equation (11).

H_MV_offset=(v _(1x) −v _(0x))*N/w, (v _(1y) −v _(0y))*N/w;

V_MV_offset=(v _(2x) −v _(0x))*N/h, (v _(2y) −v _(0y))* N/h;   (11)

where w and h in Equations (9) and (11) are the width and height of thecurrent block coded in affine mode.

In the case of assigning a motion vector at a center pixel of an N×Nsub-block as a motion vector of the control point, the denominator inEquations (6) and (8) is decreased by N. For example, Equation (6) maybe rewritten as follow.

$\begin{matrix}{{V_{0{\_ x}} = {V_{B\; 0^{\prime}{\_ x}} + {\left( {V_{B\; 2^{\prime}{\_ x}} - V_{B\; 0^{\prime}{\_ x}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 0^{\prime}{\_ Y}}} \right)/(N)}} + {\left( {V_{B\; 1^{\prime}{\_ x}} - V_{B\; 0^{\prime}{\_ x}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 0^{\prime}{\_ X}}} \right)/\left( {{RefPU\_ width} - N} \right)}}}}{V_{0{\_ y}} = {V_{B\; 0^{\prime}{\_ y}} + {\left( {V_{B\; 2^{\prime}{\_ y}} - V_{B\; 0^{\prime}{\_ y}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 0^{\prime}{\_ Y}}} \right)/(N)}} + {\left( {V_{B\; 1^{\prime}{\_ y}} - V_{B\; 0^{\prime}{\_ y}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 0^{\prime}{\_ X}}} \right)/\left( {{RefPU\_ width} - N} \right)}}}}} & (12)\end{matrix}$

The third embodiment stores affine parameters such as the horizontal andvertical direction MV offsets for the coded block. The coded block maybe either a fixed size M×M block or a coding unit (CU). The size of thefixed size M×M block may depend on the minimum size allowed for applyingaffine motion prediction. In an example, M is 8 if the smallest affineInter mode or affine Merge mode block size is 8×8. For each M×M block orfor each CU, the affine parameters including a horizontal direction MVoffset (V_(1x)−V_(0x))*N/w, (V_(1y)−V_(0y))*N/w and one motion vector ofan N×N sub-block such as Mv0 (V_(0x),V_(0y)) are stored for thefour-parameter affine motion model that uses upper-left and upper-rightcontrol points. The affine parameters including a vertical direction MVoffset (V_(2x)−V_(0x))*N/h, (V_(2y)−V_(0y))*N/h and one motion vector ofan N×N sub-block such as Mv0 (V_(0x),V_(0y)) are stored for thefour-parameter affine motion model that uses upper-left and lower-leftcontrol points. If the six-parameter affine motion model that usesupper-left, upper-right, and lower-left control points is applied, theaffine parameters including both the horizontal direction MV offset(V_(1x)−V_(0x))*N/w, (V_(1y)−V_(0y))*N/w and vertical direction MVoffset (V_(2x)−V_(0x))*N/h, (V_(2y)−V_(0y))*N/h, as well as one motionvector of an N×N sub-block such as Mv0 (V_(0x),V_(0y)) are stored. Theaffine motion vectors in the affine candidate may be derived by thestored affine parameters of one or more M×M blocks or CUs correspondingto the neighboring block.

In order to preserve the precision, the horizontal or vertical MV offsetis multiplied by a scale number, where the scale number may be apredefined number or the scale number may set equal to the largestcoding unit or the CTU size. For example, a scaled horizontal directionMV offset ((V_(1x)−V_(0x))*S/w, (V_(1y)−V_(0y))*S/w) and a scaledvertical direction MV offset ((V_(2x)−V_(0x))*S/h, (V_(2y)−V_(0y))*S/h)are stored. Some examples of the scale number S is set equal to the CTUsize or a quarter of the CTU size.

In another example, the motion vectors of two or three control points ofeach M×M block or each CU are stored. The motion vectors may be storedin a line buffer. The affine motion vectors in the affine candidate forpredicting a current block are derived by the stored motion vectors ofthe control points corresponding to the neighboring block.

FIG. 5 illustrates an exemplary flowchart for a video coding system withaffine motion compensation incorporating the third embodiment of thepresent invention. The input data associated with a current block codedor to be coded in affine mode are received by the video coding system instep S50. The video coding system calculates and stores affineparameters for the neighboring blocks in step S52 and retrieves theaffine parameters corresponding to a neighboring block of the currentblock in step S54. An affine candidate including affine motion vectorsis derived according to the retrieved affine parameters in step S56. Thevideo coding system then encodes or decodes the current block in theaffine mode using the affine candidate in step S58.

Fourth Embodiment

In the fourth embodiment, compared to the conventional temporal MVbuffer for HEVC, motion vectors in one more N×N block row and one moreN×N block column are stored in the temporal MV buffer for affine motioncompensation. The concept of the fourth embodiment is similar to thesecond embodiment, but original motion vectors of sub-blocks located inthe top row and the left-most column of one or more neighboring codedblocks are stored instead of replacing motion vectors in the sub-blockslocated closer to the current block. The temporal buffer in thisembodiment stores original motion vectors of a first neighboring N×Nblock row, a first neighboring N×N block column, a top N×N block row setand a left-most N×N block column set. The first neighboring N×N blockrow is a last row in one or more upper neighboring blocks that isclosest to a top boundary of the current block, and the firstneighboring N×N block column is a last column in one or more leftneighboring blocks that is closest to a left boundary of the currentblock. The top N×N block row set comprises a first row in one or moreupper neighboring blocks and the left-most N×N block column setcomprises a first column in one or more left neighboring blocks.

FIG. 6 illustrates an example of affine candidate derivation by storingmotion vectors of 2 N×N block rows and 2 N×N block columns. As shown inFIG. 6, original motion vectors V_(A1) and V_(B2) of sub-blocks 614 and626 in a first N×N block column adjacent to the left block boundary of acurrent block 60 and a first N×N block row above the current block 60are stored in the buffer. The first N×N block column and the first N×Nblock row in neighboring blocks 61, 62 and 63 are the closest column androw to the current block 60. Original motion vectors V_(A2), V_(A0),V_(E2), V_(E0), V_(E1), V_(B0), and V_(B1) of sub-blocks 616, 612, 636,632, 634, 622 and 624 in the left-most N×N block column of neighboringblocks 61 and 62 or the top N×N block row of neighboring blocks 62 and63 are also stored in the buffer. For example, these original motionvectors V_(A2), V_(A0), V_(E2), V_(E0), V_(E1), V_(B0), and V_(B1) ofthe left-most N×N block column or the top N×N block row of theneighboring blocks overwrite the buffer space for storing motion vectorsof sub-blocks 615, 613, 637, 633, 635, 623 and 625 in a second N×N blockrow and a second N×N block column. In this embodiment, the originalmotion vectors at the control points of neighboring blocks are stored inthe temporal MV buffer with an overhead of only one additional MV rowand one additional MV column.

FIG. 7 illustrates an exemplary flowchart for a video coding system withaffine motion compensation incorporating the second or the fourthembodiments of the present invention. The input data associated with acurrent block in a current picture is received at a video encoder sideor a video bitstream corresponding to compressed data including thecurrent block is received at a video decoder side in step S70. Step S72retrieves two or more motion vectors of neighboring block from atemporal buffer, where at least one retrieved motion vector is notcorresponding to a sub-block in a first neighboring N×N block row or afirst neighboring N×N block column of the current block. The videocoding system derives an affine candidate using the retrieved motionvectors in step S74, and encodes or decodes the current block bypredicting the current block using one or more motion compensated blocksby the derived affine candidate in step S76. The affine motion vectorspredict movements between points of the current block and correspondingpoints of the one or more motion compensated block. The temporal bufferstores selective motion vectors less than all motion vectors of codedblocks in the current picture. At least one of the retrieved motionvectors is a replacing motion vector according to the second embodiment,and all the retrieved motion vectors are original motion vectors atcontrol points of the neighboring block according to the fourthembodiment.

Fifth Embodiment

In the fifth embodiment, the video coding system with affine motioncompensation reuses the conventional temporal MV buffer required by theHEVC standard so no additional buffer is required. In other words,affine motion prediction only needs motion vectors of one neighboringN×N block row and one neighboring N×N block column in this embodiment.FIG. 8 illustrates an example of affine candidate derivation usingmotion vectors of one neighboring N×N block row and one neighboring N×Nblock column. As shown in FIG. 8, two motion vectors in the closest N×Nblock row or the closest N×N block column in a neighboring block of acurrent block 80 are used to derive an affine candidate for the currentblock. For example, when the neighboring block 82 above the currentblock 80 is selected for deriving the affine candidate, motion vectorsV_(B2) and V_(B3) in sub-blocks 826 and 828 are retrieved for derivingaffine motion vectors in the affine candidate according to afour-parameter affine motion model; when the neighboring block 81 on theleft side of the current block 80 is selected for deriving the affinecandidate, motion vectors V_(A1) and V_(A3) in sub-blocks 814 and 818are retrieved for deriving affine motion vectors in the affine candidateaccording to the four-parameter affine motion model. The neighboringblock 83 adjacent to an upper-left corner of the current block 80 is nota valid neighboring block for affine candidate derivation in thisembodiment.

Equation (13) demonstrates an example of modifying Equation (8) forderiving the affine motion vectors Mv0 and Mv1 in the affine candidatefrom the motion vectors V_(B2) and V_(B3) in the neighboring block 82according to a four-parameter affine motion model.

$\begin{matrix}{{{V_{0{\_ x}} = {V_{B\; 2{\_ x}} - {\left( {V_{B\; 3{\_ y}} - V_{B\; 2{\_ y}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 2{\_ Y}}} \right)/{RefPU}_{B}}{\_ width}} + {\left( {V_{B\; 3{\_ x}} - V_{B\; 2{\_ x}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 2{\_ X}}} \right)/{RefPU}_{B}}{\_ width}}}},{{V_{0{\_ y}} = {V_{B\; 2{\_ y}} + {\left( {V_{B\; 3{\_ x}} - V_{B\; 2{\_ x}}} \right)*{\left( {{posCurPU\_ Y} - {{posB}\; 2{\_ Y}}} \right)/{RefPU}_{B}}{\_ width}} + {\left( {V_{B\; 3{\_ y}} - V_{B\; 2{\_ y}}} \right)*{\left( {{posCurPU\_ X} - {{posB}\; 2{\_ X}}} \right)/{RefPU}_{B}}{\_ width}}}};}}\mspace{20mu} {{V_{1{\_ x}} = {V_{0{\_ x}} + {\left( {V_{B\; 3{\_ x}} - V_{B\; 2{\_ x}}} \right)*{{PU\_ width}/{RefPU}_{B}}{\_ width}}}},\mspace{20mu} {{V_{1{\_ y}} = {V_{0{\_ y}} + {\left( {V_{B\; 3{\_ y}} - V_{B\; 2{\_ y}}} \right)*{{PU\_ width}/{RefPU}_{B}}{\_ width}}}};}}} & (13)\end{matrix}$

where (V₀ _(_) _(x), V₀ _(_) _(y)) represents the motion vector Mv0 onthe upper-left corner of the current block 80, and (V₁ _(_) _(x), V₁_(_) _(y)) represents the motion vector Mv1 on the upper-right corner ofthe current block 80. A coordinate (posCurPU_X, posCurPU_Y) represents apixel position of an upper-left corner sample of the current block 80relative to an upper-left corner sample of the current picture. Acoordinate (posB2_X, posB2_Y) represents a pixel position of anupper-left sample of the sub-block 826 relative to the upper-left cornersample of the current picture. RefPU_(B) _(_)width represents the widthof the neighboring block 82 and PU_width represents the width of thecurrent block 80.

The sixth to ninth embodiments described in the following are related toimplementing Adaptive Motion Vector Resolution (AMVR) with affine motioncompensation. AMVR provides a flexible scheme to reduce motion vectordifference (MVD) transmission overhead by adaptively restricting MVD tointeger pixel resolution. An AMVR flag is signaled for a coding unit(CU) or a prediction unit (PU) to indicate whether integer pixelresolution is used or fractional pixel resolution is used. Animplementation of affine motion compensation disclosed in the literatureproposes not to signal the AMVR flag for CU or PU coded in an affinemode, so AMVR is always disabled for affine coded blocks. Affine motionvectors for all control points in an affine-coded block are infractional pixel resolution by default.

Sixth Embodiment

In the sixth embodiment, adaptive motion vector resolution is enabledwith affine motion compensation and an AMVR flag is signaled for eachaffine-coded block. The AMVR flag in one example only controls theresolution of MVDs for the affine-coded block to be integer pixelresolution or fractional pixel resolution. The AMVR flag in anotherexample controls the resolution of MVDs as well as the resolution ofmotion vector predictors (MVPs) for the affine-coded block so the finalmotion vectors are in integer resolution if the AMVR flag indicates aninteger pixel resolution is used. For a current block coded in an affinemode such as affine Inter mode, there are M MVDs calculated from Mcorresponding MVPs, where M refers to the number of control points forthe current block. M in this example is selected from 0, 1, 2, 3, and 4.The M corresponding MVPs are affine motion vectors in an affinecandidate, and each affine motion vector is a predictor for a motionvector at one control point for the affine-coded block. If the currentblock is coded in affine Inter mode and the AMVR flag is true,indicating integer pixel resolution is used for the current block, theMVDs of the control points are in integer pixel resolution. In the casewhere the AMVR flag is also used to regulate the resolution of MVPs, allthe MVPs associated with the MVDs of the control points are also roundedto integer pixel resolution if the AMVR flag is true. In the case wherethe AMVR flag is only used to regulate the resolution of MVDs, the MVPsassociated with the MVDs of the control points may be fractional pixelresolution when the AMVR flag is true. For those control points withoutan associated MVD, that is when the MVD is inferred to be zero, the MVPsfor the control points may still be in fractional pixel resolution.

Seventh Embodiment

In the seventh embodiment, MVDs are signaled before a corresponding AMVRflag in syntax design, and for an affine coded block, if there is atleast one non-zero MVD for the control points of the affine coded block,the pixel resolution of the at least one non-zero MVD or at least onedecoded motion vector is determined according to the AMVR flag. Thedecoded motion vector is derived by summing the MVD and a correspondingMVP in an affine candidate selected for the affine coded block. If theMVDs for all control points of the affine coded block are zero, the MVPsof the control points may remain in fractional pixel resolution, and theAMVR flag is not necessary to be signaled in the video bitstream.

Eighth Embodiment

In this embodiment, a block coded in affine Inter mode is restricted touni-directional prediction by disabling bi-directional prediction toreduce the system complexity and MVD overhead. For example, if an affineflag indicates a current block is coded or to be coded in affine Intermode, an Inter prediction direction interDir for the current block isset to either 0 or 1, where 0 indicates List 0 uni-directionalprediction and 1 indicates List 1 uni-directional prediction. In oneexample of the eighth embodiment, bi-directional prediction is allowedfor an affine Inter coded block only if MVDs of the affine Inter codedblock are in integer pixel resolution. In other words, enabling ordisabling of bi-directional prediction for an affine Inter coded blockdepends on the value of an AMVR flag for the affine Inter coded block.The MVD overhead is relatively small when an AMVR flag indicates theMVDs of the affine Inter coded block is in integer pixel resolution, sobi-directional prediction is allowed for the affine Inter coded block.

Ninth Embodiment

A video coding method or a video coding system implementing the ninthembodiment determines the motion vector resolution for a current blockaccording to an Inter prediction direction interDir of the current blockand whether the current block is coded in affine Inter mode. In thisembodiment, the Inter prediction direction interDir and an affine flagfor the current block are signaled prior to an AMVR flag for the currentblock in the CU syntax structure, so when interDir is equal to 2,indicating bi-direction prediction is applied for the current block, andthe affine flag is true, indicating affine Inter mode is used, the AMVRflag is inferred to be true and not necessary to be signaled for thecurrent block.

FIG. 9 illustrates an exemplary system block diagram for a Video Encoder900 based on High Efficiency Video Coding (HEVC) with affine motioncompensation according to an embodiment of the present invention. IntraPrediction 910 provides intra predictors based on reconstructed videodata of a current picture, whereas Affine Prediction 912 performs motionestimation (ME) and motion compensation (MC) according to an affinemotion model to provide predictors based on video data from otherpictures. Each block in the current picture processed by AffinePrediction 912 selects to be encoded in affine Inter mode by AffineInter Prediction 9122 or to be encoded in affine Merge mode by AffineMerge prediction 9124. For a block encoded in affine Inter mode oraffine Merge mode, a final affine candidate is selected from one or moreaffine candidate to derive one or more compensated blocks using theaffine motion model derived by the final affine candidate, and the oneor more compensated blocks are used to predict the block. The affinecandidate may be derived using one of the embodiments with a temporal MVbuffer which stores selective motion vectors less than all motionvectors of coded blocks in the current picture. The Affine MergePrediction 9124 constructs one or more affine Merge candidates accordingto motion vectors of one or more neighboring coded blocks and insertsthe one or more affine Merge candidates in a Merge candidate list.Affine Merge mode allows the inheritance of affine motion vectors atcontrol points from the neighboring coded block; therefore motioninformation is only signaled by a merge index. The merge index forselecting the final affine candidate is then signaled in an encodedvideo bitstream. The affine Merge candidate may be derived using one ofthe embodiments of the present invention. For a block coded in affineInter mode, motion information such as Motion vector difference (MVD)between the affine motion vectors in the final affine candidate andmotion vectors at control points of the block are coded in the encodedvideo bitstream. A resolution of the MVD for the block coded in affineInter mode is either integer pixel resolution or fractional pixelresolution according to an embodiment of the present invention. Switch914 selects one of the outputs from Intra Prediction 910 and AffinePrediction 912 and supplies the selected predictor to Adder 916 to formprediction errors, also called prediction residual signal.

The prediction residual signal is further processed by Transformation(T) 918 followed by Quantization (Q) 920. The transformed and quantizedresidual signal is then coded by Entropy Encoder 934 to form the encodedvideo bitstream. The encoded video bitstream is then packed with sideinformation such as the Merge index and the MVD. The data associatedwith the side information are also provided to Entropy Encoder 934. Whenmotion compensation prediction mode is used, a reference picture orpictures have to be reconstructed at the encoder end as well. Thetransformed and quantized residual signal is processed by InverseQuantization (IQ) 922 and Inverse Transformation (IT) 924 to recover theprediction residual signal of the reference picture or pictures. Asshown in FIG. 9, the prediction residual signal is recovered by addingback to the selected predictor at Reconstruction (REC) 926 to producereconstructed video data. The reconstructed video data may be stored inReference Picture Buffer (Ref. Pict. Buffer) 932 and used for predictionof other pictures. The reconstructed video data from REC 926 may besubject to various impairments due to the encoding processing,consequently, in-loop processing Deblocking Filter (DF) 928 and SampleAdaptive Offset (SAO) 930 is applied to the reconstructed video databefore storing in the Reference Picture Buffer 932 to further enhancepicture quality. DF information from DF 928 and SAO information from SAO930 are also provided to Entropy Encoder 934 for incorporation into theencoded video bitstream. The temporal MV buffer for storing motionvectors for affine candidate derivation may be implemented in Ref. Pict.Buffer 932 or any other memory coupling with the Affine Prediction 912.

A corresponding Video Decoder 1000 for the Video Encoder 900 of FIG. 9is shown in FIG. 10. The encoded video bitstream is the input to theVideo Decoder 1000 and is decoded by Entropy Decoder 1010 to recover thetransformed and quantized residual signal, DF and SAO information, andother system information. The decoding process of Decoder 1000 issimilar to the reconstruction loop at the Encoder 900, except Decoder1000 only requires Motion Compensation (MC) 1014 in Affine Prediction1014. Affine Prediction 1014 includes Affine Inter Prediction 10142 andAffine Merge Prediction 10144. Blocks coded in affine Inter mode isdecoded by Affine Inter Prediction 10142 and blocks coded in affineMerge mode is decoded by Affine Merge Prediction 10144. A final affinecandidate is selected for a block coded in affine Inter mode or affineMerge mode, and one or more compensated blocks are derived according tothe final affine candidate. The final affine candidate may be derivedaccording to one of the embodiments of the present invention with atemporal MV buffer storing selective MVs less than all MVs of codedblocks in the current picture. Switch 1016 selects intra predictor fromIntra Prediction 10512 or affine predictor from Affine Prediction 1014according to decoded mode information. The transformed and quantizedresidual signal is recovered by Inverse Quantization (IQ) 1020 andInverse Transformation (IT) 1022. The recovered transformed andquantized residual signal is reconstructed by adding back the predictorin REC 1018 to produce reconstructed video. The reconstructed video isfurther processed by DF 1024 and SAO 1026 to generate final decodedvideo. If the currently decoded picture is a reference picture, thereconstructed video of the currently decoded picture is also stored inRef. Pict. Buffer 1028. The temporal MV buffer storing motion vectorsfor affine candidate derivation may be implemented in Ref. Pict. Buffer1028 or any other memory coupling to Affine Prediction 1014.

Various components of the Video Encoder 900 and the Video Decoder 1000in FIG. 9 and FIG. 10 and various video coding process described in thedescribed embodiments may be implemented by hardware components, one ormore processors configured to execute program instructions stored in amemory, or a combination of hardware and processor. For example, aprocessor executes program instructions to control receiving of inputdata associated with a current block. The processor is equipped with asingle or multiple processing cores. In some examples, the processorexecutes program instructions to perform functions in some components inthe Encoder 900 and the Decoder 1000, and the memory electricallycoupled with the processor is used to store the program instructions,information corresponding to the affine modes, reconstructed images ofblocks, and/or intermediate data during the encoding or decodingprocess. The memory in some embodiment includes a non-transitorycomputer readable medium, such as a semiconductor or solid-state memory,a random access memory (RAM), a read-only memory (ROM), a hard disk, anoptical disk, or other suitable storage medium. The memory may also be acombination of two or more of the non-transitory computer readablemedium listed above. As shown in FIGS. 9 and 10, the Encoder 900 and theDecoder 1000 may be implemented in the same electronic device, sovarious functional components of the Encoder 900 and Decoder 1000 may beshared or reused if implemented in the same electronic device. Forexample, one or more of Reconstruction 926, Transformation 918,Quantization 920, Deblocking Filter 928, Sample Adaptive Offset 930, andReference Picture Buffer 932 in FIG. 9 may also be used to function asReconstruction 1018, Transformation 1022, Quantization 1020, DeblockingFilter 1024, Sample Adaptive Offset 1026, and Reference Picture Buffer1028 in FIG. 10, respectively. In some example, a portion of IntraPrediction 910 and Affine Prediction 912 in FIG. 9 may share or reused aportion of Intra Prediction 1012 and Affine Prediction 1014 in FIG. 10.

Although the first embodiment to the ninth embodiment of video codingmethod with affine motion compensation are described, the invention isnot limited to these embodiments. In each embodiment, the choice of thevideo coding method with affine motion compensation is an example toillustrate various embodiments and should not be understood as arestriction or a requirement for any embodiment of the invention. Theabove description is presented to enable a person of ordinary skill inthe art to practice the present invention as provided in the context ofa particular application and its requirement. Various modifications tothe described embodiments will be apparent to those with skill in theart, and the general principles defined herein may be applied to otherembodiments. Therefore, the present invention is not intended to belimited to the particular embodiments shown and described, but is to beaccorded the widest scope consistent with the principles and novelfeatures herein disclosed. In the above detailed description, variousspecific details are illustrated in order to provide a thoroughunderstanding of the present invention. Nevertheless, it will beunderstood by those skilled in the art that the present invention may bepracticed.

The invention may be embodied in other specific forms without departingfrom its spirit or essential characteristics. The described examples areto be considered in all respects only as illustrative and notrestrictive. The scope of the invention is therefore, indicated by theappended claims rather than by the foregoing description. All changeswhich come within the meaning and range of equivalency of the claims areto be embraced within their scope.

1. A method of video coding with affine motion compensation in a videocoding system, comprising: receiving input data associated with acurrent block in a current picture at a video encoder or receiving avideo bitstream corresponding to compressed data including the currentblock in the current picture at a video decoder, wherein the currentblock is coded or to be coded in an affine mode according to an affinemotion model; retrieving two or more motion vectors of a neighboringblock from a temporal buffer for the current block, wherein at least oneof the retrieved motion vectors is not corresponding to a sub-block in afirst neighboring N×N block row closest to a top boundary of the currentblock or a first neighboring N×N block column closest to a left boundaryof the current block, wherein the temporal buffer stores selectivemotion vectors less than all motion vectors of previous coded blocks inthe current picture, and N×N is a block size for storing one motionvector in the temporal buffer; deriving an affine candidate includingaffine motion vectors using the retrieved two or more motion vectors ofthe neighboring block; and encoding or decoding the current block bypredicting the current block using one or more motion compensated blocksby the derived affine candidate, wherein the affine motion vectorspredict a movement between a point of the current block and acorresponding point of the one or more motion compensated blocks.
 2. Themethod of claim 1, wherein the temporal buffer stores motion vectors of2 N×N block rows and 2 N×N block columns, and the 2 N×N block rowsinclude the first neighboring N×N block row and a second neighboring N×Nblock row that are closest to the top boundary of the current block, andthe 2 N×N block columns include the first neighboring N×N block columnand a second neighboring N×N block column closest to the left boundaryof the current block.
 3. The method of claim 1, wherein the retrievedtwo or more motion vectors includes first and second motion vectors, thefirst motion vector is a replacing motion vector replacing an originalmotion vector at an upper-left corner of the neighboring block and thesecond motion vector is a replacing motion vector replacing an originalmotion vector at an upper-right corner of the neighboring block if theneighboring block is above the current block or adjacent to anupper-left corner of the current block; wherein the first motion vectoris a replacing motion vector replacing an original motion vector at anupper-left corner of the neighboring block and the second motion vectoris an original motion vector at an upper-right corner of the neighboringblock if the neighboring block is on the left side of the current block.4. The method of claim 1, wherein the retrieved two or more motionvectors includes first and second motion vectors of first and secondsub-blocks in the neighboring block, said deriving an affine candidatefurther comprises: deriving a first affine motion vector of the affinemotion vectors using the first and second motion vectors, a pixelposition of the current block relative to the current picture, a pixelposition of the first sub-block relative to the current picture, and awidth of the neighboring block; and deriving a second affine motionvector using the first and second motion vectors, the width of theneighboring block, and a width of the current block.
 5. The method ofclaim 1, wherein the retrieved two or more motion vectors includesfirst, second and third motion vectors of first, second and thirdsub-blocks in the neighboring block, wherein deriving the affinecandidate further comprises: deriving a first affine motion vector ofthe affine motion vectors using at least two of the first, second andthird motion vectors, a pixel position of the current block relative tothe current picture, a pixel position of the first sub-block relative tothe current picture, and a width of the neighboring block; deriving asecond affine motion vector using the first and second motion vectors,the width of the neighboring block, and a width of the current block;and deriving a third affine motion vector using the first [[nd]] andthird motion vectors and at least one of a height of the neighboringblock and a height of the current block.
 6. The method of claim 2,wherein the retrieved two or more motion vectors includes first, secondand third motion vectors, the first motion vector is a replacing motionvector replacing an original motion vector at an upper-left corner ofthe neighboring block, the second motion vector is a replacing motionvector replacing an original motion vector at an upper-right corner ofthe neighboring block, and the third motion vector is an original motionvector at a lower-left corner of the neighboring block if theneighboring block is above the current block; wherein the first motionvector is a replacing motion vector replacing an original motion vectorat an upper-left corner of the neighboring block, the second motionvector is a replacing motion vector replacing an original motion vectorat an upper-right corner of the neighboring block, and the third motionvector is a replacing motion vector replacing an original motion vectorat a lower-left corner of the neighboring block if the neighboring blockis adjacent to an upper-left corner of the current block; and whereinthe first motion vector is a replacing motion vector replacing anoriginal motion vector at an upper-left corner of the neighboring block,the second motion vector is an original motion vector at an upper-rightcorner of the neighboring block, and the third motion vector is areplacing motion vector replacing an original motion vector at alower-left corner of the neighboring block if the neighboring block ison the left side of the current block.
 7. The method of claim 1, whereinthe current block is predicted by said one or more motion compensatedblocks by the affine motion vectors in the affine candidate according toa four-parameter affine motion model with two control points oraccording to a six-parameter affine motion model with three controlpoints.
 8. The method of claim 1, wherein the temporal buffer stores MN×N block rows above the current block, and M is less than a largestcoding unit height divided by N.
 9. The method of claim 1, wherein thetemporal buffer stores K N×N block columns on a left side of the currentblock, and K is equal to less than a largest coding unit width dividedby N.
 10. The method of claim 1, wherein the retrieved two or moremotion vectors are original motion vectors at two or more control pointsof the neighboring block.
 11. The method of claim 10, wherein thecontrol points of the neighboring block include at least two of anupper-left corner, an upper-right corner, a bottom-right corner, and abottom-left corner of the neighboring block.
 12. The method of claim 10,wherein the temporal buffer stores original motion vectors of a firstneighboring N×N block row, a first neighboring N×N block column, a topN×N block row set and a left-most N×N block column set, where the firstneighboring N×N block row is a last row in at least one upperneighboring block that is closest to the top boundary of the currentblock, the first neighboring N×N block column is a last column in atleast one left neighboring block that is closest to the left boundary ofthe current block, the top N×N block row set comprises a first row inthe at least one upper neighboring block, and the left-most N×N blockcolumn set comprises a first column in the at least one left neighboringblock.
 13. (canceled)
 14. (canceled)
 15. (canceled)
 16. (canceled) 17.(canceled)
 18. A method of video coding with affine motion compensationin a video coding system, comprising: receiving input data associatedwith a current block in a current picture at a video encoder orreceiving a video bitstream corresponding to compressed data includingthe current block in the current picture at a video decoder, wherein thecurrent block is coded or to be coded in an affine mode according to anaffine motion model; retrieving two or more motion vectors of a validneighboring block from a temporal buffer for the current block, whereinthe temporal buffer stores motion vectors of one neighboring N×N blockrow or one neighboring N×N block column of the current block, and N×N isa block size for storing one motion vector in the temporal buffer;deriving an affine candidate including affine motion vectors using theretrieved two or more motion vectors of the valid neighboring block; andencoding or decoding the current block by predicting the current blockusing one or more motion compensated blocks by the derived affinecandidate, wherein the affine motion vectors predict a movement betweena point of the current block and a corresponding point of the one ormore motion compensated blocks.
 19. The method of claim 18, wherein theretrieved two or more motion vectors correspond to two motion vectors,one of said two motion vectors is an original motion vector at anlower-left corner of the valid neighboring block and the other one ofsaid two motion vectors is an original motion vector at an lower-rightcorner of the valid neighboring block if the valid neighboring block isabove the current block.
 20. The method of claim 18, wherein the currentblock is predicted by said one or more motion compensated blocks by theaffine motion vectors in the affine candidate according to afour-parameter affine motion model with two control points.
 21. Anapparatus of video coding with affine motion compensation in a videocoding system, the apparatus comprising one or more electronic circuitsconfigured for: receiving input data associated with a current block ina current picture at a video encoder or receiving a video bitstreamcorresponding to compressed data including the current block in thecurrent picture at a video decoder, wherein the current block is codedor to be coded in an affine mode according to an affine motion model;retrieving two or more motion vectors of a neighboring block from atemporal buffer for the current block, wherein at least one of theretrieved motion vectors is not corresponding to a sub-block in a firstneighboring N×N block row closest to a top boundary of the current blockor a first neighboring N×N block column closest to a left boundary ofthe current block, wherein the temporal buffer stores selective motionvectors less than all motion vectors of previous coded blocks in thecurrent picture, and N×N is a block size for storing one motion vectorin the temporal buffer; deriving an affine candidate including affinemotion vectors using the retrieved two or more motion vectors of theneighboring block; and encoding or decoding the current block bypredicting the current block using one or more motion compensated blocksby the derived affine candidate, wherein the affine motion vectorspredict a movement between a point of the current block and acorresponding point of the one or more motion compensated blocks. 22.(canceled)
 23. An apparatus of video coding with affine motioncompensation in a video coding system, the apparatus comprising one ormore electronic circuits configured for: receiving input data associatedwith a current block in a current picture at a video encoder orreceiving a video bitstream corresponding to compressed data includingthe current block in the current picture at a video decoder, wherein thecurrent block is coded or to be coded in an affine mode according to anaffine motion model; retrieving two or more motion vectors of a validneighboring block from a temporal buffer for the current block, whereinthe temporal buffer stores motion vectors of one neighboring N×N blockrow or one neighboring N×N block column of the current block, and N×N isa block size for storing one motion vector in the temporal buffer;deriving an affine candidate including affine motion vectors using theretrieved two or more motion vectors of the valid neighboring block; andencoding or decoding the current block by predicting the current blockusing one or more motion compensated blocks by the derived affinecandidate, wherein the affine motion vectors predict a movement betweena point of the current block and a corresponding point of the one ormore motion compensated blocks.
 24. The method of claim 18, wherein theretrieved two or more motion vectors correspond to two motion vectors,one of said two motion vectors is an original motion vector at anupper-right corner of the valid neighboring block and the other one ofsaid two motion vectors is an original motion vector at an lower-rightcorner of the valid neighboring block if the valid neighboring block ison the left side of the current block.